Systems and methods for enhanced image adaptation

ABSTRACT

Systems and methods for adapting images for substantially optimal presentation by heterogeneous client display sizes are described. In one aspect, an image is modeled with respect to multiple visual attentions to generate respective attention objects for each of the visual attentions. For each of one or more image adaptation schemes, an objective measure of information fidelity (IF) is determined for a region R of the image. The objective measures are determined as a function of a resource constraint of the display device and as a function of a weighted sum of IF of each attention object in the region R. A substantially optimal adaptation scheme is then selected as a function of the calculated objective measures. The image is then adapted via the selected substantially optimal adaptation scheme to generate an adapted image as a function of at least the target area of the client display.

RELATED APPLICATIONS

This patent application is related to:

-   U.S. patent application Ser. No. 10/286,053, titled “Systems and    Methods for Generating a Comprehensive User Attention Model”, filed    on Nov. 1, 2002, commonly assigned herewith, and hereby incorporated    by reference; and,-   U.S. patent application Ser. No. 10/285,933, titled “Systems and    Methods for Generating a Motion Attention Model”, filed on Nov. 1,    2002, commonly assigned herewith, and hereby incorporated by    reference.

TECHNICAL FIELD

The invention pertains to image processing.

BACKGROUND

Advances in hardware and software have resulted in numerous new types ofmobile Internet client devices such as hand-held computers, personaldigital assistants (PDAs), telephones, etc. As a trade-off for compactdesign and mobility, such client devices (hereinafter often referred toas “small-form-factor” devices) are generally manufactured to operate inresource constrained environments. A resource constrained environment isa hardware platform that provides substantially limited processing,memory, and/or display capabilities as compared, for example, to adesktop computing system. As a result, the types of devices across whichInternet content may be accessed and displayed typically have diversecomputing and content presentation capabilities as compared to oneanother.

Internet content authors and providers generally agree that serving aclient base having disparate computing and content presentationcapabilities over networks having different data throughputcharacteristics presents a substantial challenge. Conventional imageadaptation techniques attempt to meet this challenge by reducing thesize of high-resolution Internet content via resolution and contentreduction as well as data compression techniques. Unfortunately, eventhough employing such conventional image adaptation techniques mayspeed-up content delivery to the client over a low-bandwidth connection,excessively reduced and compressed content often provide Internet clientdevice users with a viewing experience that is not consistent with humanperception. Such a viewing experience is also often contrary to thehigh-quality impression that content authors/providers prefer for theviewer to experience, and contrary to the universal access to highquality images viewers generally desire.

To make matters worse, algorithms used in conventional image adaptationschemes often involve large number of adaptation rules or over-intensivecomputations (e.g., semantic analysis) that are impracticable forsystems that provide on-the-fly adaptive content delivery for mobilesmall-form-factor devices.

The following arrangements and procedures address these and otherproblems of conventional techniques to adapt content for delivery andpresentation by Internet client devices.

SUMMARY

Systems and methods for adapting images for substantially optimalpresentation by heterogeneous client display sizes are described. In oneaspect, an image is modeled with respect to multiple visual attentionsto generate respective attention objects for each of the visualattentions. For each of one or more image adaptation schemes, anobjective measure of information fidelity (IF) is determined for aregion R of the image. The objective measures are determined as afunction of a resource constraint of the display device and as afunction of a weighted sum of IF of each attention object in the regionR. A substantially optimal adaptation scheme is then selected as afunction of the calculated objective measures. The image is then adaptedvia the selected substantially optimal adaptation scheme to generate anadapted image as a function of at least the target area of the clientdisplay.

BRIEF DESCRIPTION OF THE DRAWINGS

The following detailed description is described with reference to theaccompanying figures. In the figures, the left-most digit of a componentreference number identifies the particular figure in which the componentfirst appears.

FIG. 1 is a block diagram of an exemplary computing environment withinwhich systems and methods for enhanced image adaptation may beimplemented.

FIG. 2 shows further exemplary aspects of application programs andprogram data of FIG. 1 used for enhanced image adaptation.

FIG. 3 shows exemplary results of a conventional image adaptation asdisplayed by a client device.

FIG. 4 shows an exemplary adapted image that was generated byattention-based modeling of an image adapting device of the invention.

FIG. 5 shows an exemplary adapted image that was generated byattention-based modeling of an original image.

FIG. 6 shows that image attention objects are segmented into regions andrespectively adapted to presentation characteristics of a client displaytarget area.

FIG. 7 shows a binary tree to illustrate a branch and bound process toidentify an optimal image adaptation solution.

FIG. 8 shows an exemplary procedure for enhanced image adaptation inview of client resource constraints.

DETAILED DESCRIPTION

Overview

The following described arrangements and procedures provide a frameworkto adapt a static image in view of a resource constrained client such asa small-form-factor client for display. To this end, the frameworkanalyzes the image in view of multiple computational visual attentionmodels. As a basic concept, “attention” is a neurobiologicalconcentration of mental powers upon an object; a close or carefulobserving or listening, which is the ability or power to concentratementally. The multiple visual attention models used in this frameworkare computational to dynamically reduce attention into a series oflocalized algorithms. Results from visual attention modeling of theimage are integrated to identify one or more portions, or “attentionobjects” (AOs) of the image that are most likely to be of interest to aviewer. A substantially optimal adaptation scheme is then selected foradapting the identified AOs. The scheme is selected as a function ofobjective measures indicating that the scheme will result in highestimage fidelity of the AOs after adaptation in view of client resourceconstraints such as target screen size. The identified image portionsare then adapted via the selected adaptation scheme for considerablyoptimal viewing at the client.

An Exemplary Operating Environment

Turning to the drawings, wherein like reference numerals refer to likeelements, the invention is illustrated as being implemented in asuitable computing environment. Although not required, the invention isdescribed in the general context of computer-executable instructions,such as program modules, being executed by a personal computer. Programmodules generally include routines, programs, objects, components, datastructures, etc., that perform particular tasks or implement particularabstract data types.

FIG. 1 illustrates an example of a suitable computing environment 120 onwhich the subsequently described systems, apparatuses and methods forenhanced image adaptation may be implemented. Exemplary computingenvironment 120 is only one example of a suitable computing environmentand is not intended to suggest any limitation as to the scope of use orfunctionality of systems and methods the described herein. Neithershould computing environment 120 be interpreted as having any dependencyor requirement relating to any one or combination of componentsillustrated in computing environment 120.

The methods and systems described herein are operational with numerousother general purpose or special purpose computing system environmentsor configurations. Examples of well known computing systems,environments, and/or configurations that may be suitable include, butare not limited to, including hand-held devices, multi-processorsystems, microprocessor based or programmable consumer electronics,network PCs, minicomputers, mainframe computers, portable, communicationdevices, and the like. The invention may also be practiced indistributed computing environments where tasks are performed by remoteprocessing devices that are linked through a communications network. Ina distributed computing environment, program modules may be located inboth local and remote memory storage devices.

As shown in FIG. 1, computing environment 120 includes a general-purposecomputing device in the form of a computer 130. The components ofcomputer 130 may include one or more processors or processing units 132,a system memory 134, and a bus 136 that couples various systemcomponents including system memory 134 to processor 132.

Bus 136 represents one or more of any of several types of busstructures, including a memory bus or memory controller, a peripheralbus, an accelerated graphics port, and a processor or local bus usingany of a variety of bus architectures. By way of example, and notlimitation, such architectures include Industry Standard Architecture(ISA) bus, Micro Channel Architecture (MCA) bus, Enhanced ISA (EISA)bus, Video Electronics Standards Association (VESA) local bus, andPeripheral Component Interconnects (PCI) bus also known as Mezzaninebus.

Computer 130 typically includes a variety of computer readable media.Such media may be any available media that is accessible by computer130, and it includes both volatile and non-volatile media, removable andnon-removable media. In FIG. 1, system memory 134 includes computerreadable media in the form of volatile memory, such as random accessmemory (RAM) 140, and/or non-volatile memory, such as read only memory(ROM) 138. A basic input/output system (BIOS) 142, containing the basicroutines that help to transfer information between elements withincomputer 130, such as during start-up, is stored in ROM 138. RAM 140typically contains data and/or program modules that are immediatelyaccessible to and/or presently being operated on by processor 132.

Computer 130 may further include other removable/non-removable,volatile/non-volatile computer storage media. For example, FIG. 1illustrates a hard disk drive 144 for reading from and writing to anon-removable, non-volatile magnetic media (not shown and typicallycalled a “hard drive”), a magnetic disk drive 146 for reading from andwriting to a removable, non-volatile magnetic disk 148 (e.g., a “floppydisk”), and an optical disk drive 150 for reading from or writing to aremovable, non-volatile optical disk 152 such as a CD-ROM/R/RW,DVD-ROM/R/RW/+R/RAM or other optical media. Hard disk drive 144,magnetic disk drive 146 and optical disk drive 150 are each connected tobus 136 by one or more interfaces 154.

The drives and associated computer-readable media provide nonvolatilestorage of computer readable instructions, data structures, programmodules, and other data for computer 130. Although the exemplaryenvironment described herein employs a hard disk, a removable magneticdisk 148 and a removable optical disk 152, it should be appreciated bythose skilled in the art that other types of computer readable mediawhich can store data that is accessible by a computer, such as magneticcassettes, flash memory cards, digital video disks, random accessmemories (RAMs), read only memories (ROM), and the like, may also beused in the exemplary operating environment.

A number of program modules may be stored on the hard disk, magneticdisk 148, optical disk 152, ROM 138, or RAM 140, including, e.g., anoperating system 158, one or more application programs 160, otherprogram modules 162, and program data 164.

A user may provide commands and information into computer 130 throughinput devices such as keyboard 166 and pointing device 168 (such as a“mouse”). Other input devices (not shown) may include a microphone,joystick, game pad, satellite dish, serial port, scanner, camera, etc.These and other input devices are connected to the processing unit 132through a user input interface 170 that is coupled to bus 136, but maybe connected by other interface and bus structures, such as a parallelport, game port, or a universal serial bus (USB).

A monitor 172 or other type of display device is also connected to bus136 via an interface, such as a video adapter 174. In addition tomonitor 172, personal computers typically include other peripheraloutput devices (not shown), such as speakers and printers, which may beconnected through output peripheral interface 175.

Computer 130 may operate in a networked environment using logicalconnections to one or more remote computers, such as a remote computer182. Remote computer 182 may include some or all of the elements andfeatures described herein relative to computer 130. Logical connectionsshown in FIG. 1 are a local area network (LAN) 177 and a general widearea network (WAN) 179. Such networking environments are commonplace inoffices, enterprise-wide computer networks, intranets, and the Internet.

When used in a LAN networking environment, computer 130 is connected toLAN 177 via network interface or adapter 186. When used in a WANnetworking environment, the computer typically includes a modem 178 orother means for establishing communications over WAN 179. Modem 178,which may be internal or external, may be connected to system bus 136via the user input interface 170 or other appropriate mechanism.

Depicted in FIG. 1, is a specific implementation of a WAN via theInternet. Here, computer 130 employs modem 178 to establishcommunications with at least one remote computer 182 via the Internet180.

In a networked environment, program modules depicted relative tocomputer 130, or portions thereof, may be stored in a remote memorystorage device. Thus, e.g., as depicted in FIG. 1, remote applicationprograms 189 may reside on a memory device of remote computer 182. Itwill be appreciated that the network connections shown and described areexemplary and other means of establishing a communications link betweenthe computers may be used.

FIG. 2 shows further exemplary aspects of application programs andprogram data of FIG. 1 for enhanced image adaptation. In particular,system memory 134 is shown to include application programs 160 andprogram data 164. Application programs include, for example, contentadaptation module 202 and other modules 204 such as an operating systemto provide a run-time environment, and so on. The content adaptationmodule analyzes content 206 (e.g., a Web page designed in a markuplanguage such as the Hypertext Markup Language (HTML)) in view ofmultiple visual attention models (e.g., saliency, face, text, etc.) togenerate attention data 208 for each image 210-1 through 210-K of thecontent. The attention data includes, for example, a set of attentionobjects (AOs) 212-1 through 212-N for each attention modeling schemeutilized for each image. For instance, saliency attention modelingresults in a first set of AOs for each image. Face attention modelingresults in a second set of AOs for each image, and so on.

As a result of the visual attention modeling of the images 210-1 through210-K, most perceptible information of an image will be located insidethe AOs 212-1 through 212-N. Because of this, each AO is an informationcarrier that has been computationally determined to deliver at least oneintention of the content author. For instance, an AO may represent asemantic object, such as a human face, a flower, a mobile car, text,and/or the like, that will catch part of the viewer's attention, as awhole. In light of this, the combination of information in the AOsgenerated from an image will catch most attentions of a viewer.

Each AO 212 (each one of the AOs 212-1 though 212-N) is associated withthree (3) respective attributes: a Region-Of-Interest (ROI) 214, anAttention Value (AV) 216 and a Minimal Perceptible Size (MPS) 218. TheROI is a spatial region or segment within an image that is occupied bythe particular AO. An ROI can be of any shape and ROIs of different AOsmay overlap. In one implementation, a ROI is represented by a set ofpixels in the original image. In another implementation, regular shapedROIs are denoted by their geometrical parameters, rather than with pixelsets. For example, a rectangular ROI can be defined as {Left, Top,Right, Bottom} coordinates, or {Left, Top, Width, Height} coordinates,while a circular ROI can be defined as {Center_x, Center_y, Radius}, andso on.

Different AOs 212-1 through 212-N represent different portions andamounts of an image's 210-1 through 210-K information. In light of this,the content adaptation module 202 assigns each AO in the image arespective quantified attention value (AV). Thus, an AO's AV indicatesthe relative weight of the AO's contribution to the informationcontained in the image as compared to the weights of other AOs.

The total amount of information that is available from an AO 212 is afunction of the area it occupies on an image 210-1 through 210-K. Thus,overly reducing the resolution of the AO may not include the content ormessage that the image's author originally intended for a viewer todiscern. In light of this, a Minimal Perceptible Size (MPS) 218 isassigned to each AO to indicate a minimal allowable spatial area for theAO. The content adaptation module 202 uses the MPS is used as areduction quality threshold to determine whether an AO should be furthersub-sampled or cropped during image adaptation operations.

For example, suppose an image 210 contains N number of AOs 212,{AO_(i)}, i=1, 2 . . . , N, where AO_(i) denotes the i^(−th) AO withinthe image. The MPS 218 of AO_(i) indicates the minimal perceptible sizeof AO_(i), which can be presented by the area of a scaled-down region.For instance, consider that a particular AO represents a human facewhose original resolution is 75×90 pixels. The author or publisher maydefine its MPS to be 25×30 pixels which is the smallest resolution toshow the face region without severely degrading its perceptibility. TheMPS assignment may be accomplished manually by user interaction, orcalculated automatically in view of a set of rules. In this manner, thecontent adaptation module provides end-users with an adapted image thatis not contrary to the impression that the content author/providerintended the end-user to experience.

In view of the foregoing, the content adaptation module 202 generatesAOs for each an image 210-1 through 210-K as follows:{AO_(i)}={(ROI_(i), AV_(i), MPS_(i))}, 1≦i≦N  (1).AO_(i), represents the i^(−th) AO within the image; ROI_(i) representsthe Region-Of-Interest 212 of AO_(i); AV_(i) represents the AttentionValue 214 of AO_(i); MPS_(i) represents the Minimal Perceptible Size 216of AO_(i); and, N represents the total number of AOs in the image.

For each image 210-1 through 210-K, the content adaptation module 202integrates and analyzes the attention data 208 generated from the visualattention modeling operations. Such analysis identifies a region R ofthe image that includes one or more AOs 212-1 through 212-N withsubstantially high attention values (AVs) as compared to other AOs ofthe image. (E.g., see region R 604 of FIG. 6). Thus, the identifiedregion R is objectively determined to be most likely to attract humanattention as compared to other portions of the image. The contentadaptation module then identifies a substantially optimal imageadaptation scheme (e.g., reduction, compression, and/or the like) toadapt the region R, and the image as a whole, in view of client resourceconstraints and such that the highest image fidelity (IF) over theregion R is maintained. The analyzed images are then adapted (i.e.,adapted image(s) 220) based on information from these AOs and in view ofa target client display (e.g., screen 176 or 190 of FIG. 1)characteristics. Further details of these exemplary operations are nowdescribed.

Display Constraint-Based Presentation of High Attention Value Objects

The content adaptation module 202 manipulates AOs 212 for each image 210to adapt the image 220 such that it represents as much information aspossible under target client device resource constraints such as displayscreen size, display resolution, and/or the like. However, as describedabove, each AO is reduced in size as a function not only of displayscreen size, but also as a function of the AO's quality reductionthreshold attribute, the MPS 218. Thus, although the MPS substantiallyguarantees a certain level of image quality, it is possible thatenforcement of the MPS may result in an adapted image 220 that is toolarge to view at any one time on a display screen (e.g., see displays172 and 190 of FIG. 1). In other words, such an adapted image may needto be scrolled horizontally and/or vertically for all portions of theadapted image to be presented on the client display.

In light of this, and to ensure a substantially valuable viewingexperience, the content adaptation module 202 directs the client deviceto initially present a portions of the image that includes one or moreAOs 212 that include respective AVs 214 to indicate that they are morelikely to attract the user's attention than other portions of the image.To illustrate such image adaptation in view of client resourceconstraints, please refer to the examples of FIGS. 3-5, which incombination show exemplary differences between conventional imageadaptation results (FIG. 3) and exemplary image adaptation resultsaccording to the arrangements and procedures of this invention (FIGS. 4and 5).

FIG. 3 shows exemplary results 300 of conventional image adaptation asdisplayed by a client device 300. Note that the informative text 302 inthe upper left quadrant of the adapted image is barely recognizable.This is due to the excessive resolution reduction that was performed tofit the image to a screen dimension of 240×320 pixels, which is the sizeof a typical pocket PC screen. In contrast to FIG. 3, FIG. 4 shows anexemplary adapted image 400 that was generated by the attention-basedmodeling of client device 400. Note that even though the screen sizes ofclient device 300 and client device 400 are the same, text portion 402of the image is much clearer as compared to the text 302 of FIG. 3. Thisis due to substantial optimization of the original image as a functionof attention-modeling results, and selection of those portions of theimage for presentation that have higher attention values in view of thetarget client resource constraints.

FIG. 5 shows an exemplary adapted image that was generated by theattention-based modeling of an original image. In particular, clientdevice 500 represents client device 400 of FIG. 4 in a rotated, orlandscape position. Such rotation causes the landscape viewing functionof the client display screen to activate via a client device displaydriver. Responsive to such rotation, the coordinates of the displayscreen change and the content adaptation module 202 of FIG. 2 generatesan adapted image 220 (FIG. 2) to present clear views of both the text502 and the face 504. Accordingly, the content adaptation module adaptsimages to present the most important aspect(s) of the adapted image inview of client device resource constraints (e.g., size, positionalnature/screen rotation, etc.),

Visual Attention Modeling

Turning to FIG. 2, further details of the operations used by the contentadaptation module 202 to perform visual attention modeling of theimage(s) 210-1 through 210-K are now described. In particular, thecontent adaptation module analyzes each image in view of multiple visualattention models to generate attention model data 208. In thisimplementation, the attention models include, for example, saliency,face, and text attention models. However, in another implementation, thecontent adaptation module is modified to integrate information generatedfrom analysis of the image(s) via different attention model algorithmssuch as those described in:

-   U.S. patent application Ser. No. 10/286,053, titled “Systems and    Methods for Generating a Comprehensive User Attention Model”, filed    on Nov. 1, 2002, commonly assigned herewith, and incorporated by    reference; and-   U.S. patent application Ser. No. 10/285,933, titled “Systems and    Methods for Generating a Motion Attention Model”, filed on Nov. 1,    2002, commonly assigned herewith, and incorporated by reference.

Saliency Attention Modeling

The content adaptation module 202 generates three (3) channel saliencymaps for each of the images 210-1 through 121-1. These saliency mapsrespectively identify color contrasts, intensity contrasts, andorientation contrasts. These saliency maps are represented as respectiveportions of attention data 208. Techniques to generate such maps aredescribed in “A Model of Saliency-Based Visual Attention for Rapid SceneAnalysis” by Itti et al., IEEE Transactions on Pattern Analysis andMachine Intelligence, 1998, hereby incorporated by reference.

The content adaptation module 202 then generates a final gray saliencymap, which is also represented by attention data 208, by applyingportions of the iterative method proposed in “A Comparison of FeatureCombination Strategies for Saliency-Based Visual Attention Systems, Ittiet al, Procedures of SPIE Human Vision and Electronic Imaging IV(HVEI'99), San Jose, Calif., Vol. 3644, pp. 473-82, January 1999, andhereby incorporated by reference. Saliency attention is determined viathe final saliency map as a function of the number of saliency regions,and their brightness, area, and position in the gray saliency map.

To reduce image adaptation time, the content adaptation module 202detects regions that are most attractive to human attention bybinarizing the final saliency map. Such binarization is based on thefollowing:

$\begin{matrix}{{{A\; V_{saliency}} = {\sum\limits_{({i,{j \in R}})}{B_{i,j} \cdot W_{saliency}^{i,j}}}}\;,} & (2)\end{matrix}$wherein B_(i,j) denotes the brightness of pixel point (i,j) in thesaliency region R, W_(saliency) ^(pos) ^(i,j) is the position weight ofthat pixel. Since people often pay more attention to the region near theimage center, a normalized Gaussian template centered at the image isused to assign the position weight.

The size, the position and the brightness attributes of attended regionsin the binarized or gray saliency map (attention data 208) decide thedegree of human attention attracted. The binarization threshold isestimated in an adaptive manner. Since saliency maps are representedwith arbitrary shapes with little semantic meaning. Thus, a set of MPSratios 218 are predefined for each AO 212 that represented as a saliencymap. The MPS thresholds can be manually assigned via user interaction orcalculated automatically. For example, in one implementation, the MPS ofa first region with complex textures is larger than the MPS of a secondregion with less complex texturing.

Face Attention Modeling

A person's face is generally considered to be one of the most salientcharacteristics of the person. Similarly, a dominant animal's face in avideo could also attract viewer's attention. In light of this, itfollows that the appearance of dominant faces in images 210 will attracta viewers' attention. Thus, a face attention model is applied to each ofthe images by the content adaptation module 202. Portions of attentiondata 208 are generated as a result, and include, for example, the numberof faces, their respective poses, sizes, and positions.

A real-time face detection technique is described in “StatisticalLearning of Multi-View Face Detection”, by Li et al., Proc. of EVVC2002; which is hereby incorporated by reference. In this implementation,seven (7) total face poses (with out-plane rotation) can be detected,from the frontal to the profile. The size and position of a face usuallyreflect the importance of the face. Hence,

$\begin{matrix}{{{AV}_{face} = {\sqrt{{Area}_{face}} \times W_{face}^{pos}}}\;,} & (3)\end{matrix}$wherein Area_(face) denotes the size of a detected face region andw_(face) ^(pos) is the weight of its position. In one implementation,the MPS 218 attribute of an AO 212-1 through 212-N face attention modelis a predefined absolute pixel area size. For instance, a face with anarea of 25×30 pixels in size will be visible on many different types ofdevices.

Text Attention Modeling

Similar to human faces, text regions also attract viewer attention inmany situations. Thus, they are also useful in deriving image attentionmodels. There have been so many works on text detection and recognitionand localization accuracy can reach around 90% for text larger than ten(10) points. By adopting a text detection, the content adaptation module202 finds most of the informative text regions inside images 210-1through 210-K. Similar to the face attention model, the region size isalso used to compute the attention value 214 of a text region. Inaddition, the aspect ratio of region in included in the calculation inconsideration that important text headers or titles are often in anisolated single line with large heights whose aspect ratios are quitedifferent from text paragraph blocks. In light of this, the attentionvalue for a text region is expressed as follows:AV_(text)=√{square root over (Area_(text))}×W _(AspectRatio)  (4).Area_(text) denotes the size of a detected text region, andW_(AspectRatio) is the weight of its aspect ratio generated by someheuristic rules. The MPS 218 of a text region (AO 212) can be predefinedaccording to a determined font size, which can be calculated by textsegmentation from the region size of text. For example, the MPS ofnormal text can be assigned from a specific 10 points font size inheight.

Attention Model Adjustment—Post Processing

Before integrating the multiple visual attention measurements, thecontent adaptation module 202 adjusts each AO's 212 respective attentionvalue (AV) 216. In one implementation, for purposes of simplicity, thisis accomplished via a rule-based approach. For example, respective AO AVvalues in each attention model are normalized to (0, 1), and the finalattention value is computed as follows:

$\begin{matrix}{{{AV}_{i} = {w_{k} \cdot \overset{\_}{{AV}_{i}^{k}}}},} & (5)\end{matrix}$wherein w_(k) is the weight of model k and AV_(i) ^(k) is the normalizedattention value of AO_(i) detected in the model k, e.g. saliency model,face model, text model, or any other available model.

When adapting images contained in a composite content 206 such as a Webpage, image 210 contexts are quite influential to user attention. Toaccommodate this variation in modeling image attentions, the contentadaptation module 202 implements Function-based Object Model (FOM) tounderstand a content author's intention for each object in a Web page.Such FOM is described in Chen J. L., Zhou B. Y., Shi J., Zhang H. J. andWu Q. F. (2001), Function-based Object Model Towards Website Adaptation,Proc. of the 10th Int. WWW Conf. pp. 587-596, hereby incorporated byreference. For example, images in a Web page may have differentfunctions, such as information, navigation, decoration or advertisement,etc. By using FOM analysis, the context of an image can be detected toassist image attention modeling.

Attention-Based Image Adaptation

Operations to find an optimal image adaptation of content 206 that hasbeen modeled with respect to visual attention, wherein the optimal imageadaptation is a function of resource constraints of a target clientdevice, are now described using integer programming and abranch-and-bound algorithm.

Information Fidelity

Information fidelity is the perceptual ‘look and feel’ of a modified oradapted version of content (or image), a subjective comparison with theoriginal content 206 (or image 210). The value of information fidelityis between 0 (lowest, all information lost) and 1 (highest, allinformation kept just as original). Information fidelity gives aquantitative evaluation of content adaptation that the optimal solutionis to maximize the information fidelity of adapted content underdifferent client context constraints. The information fidelity of anindividual AO 212 after adaptation is decided by various parameters suchas spatial region size, color depth, ratio of compression quality, etc.

For an image region R consisting of several AOs 212, the resultinginformation fidelity is the weighted sum of the information fidelity ofall AOs in R. Since user's attention on objects always conforms to theirimportance in delivering information, attention values of different AOsare employed as the informative weights of contributions to the wholeperceptual quality. Thus, the information fidelity of an adapted resultcan be described as

$\begin{matrix}{{IF}_{R} = {\sum\limits_{{ROI}_{i} \Subset R}{{AV}_{i} \cdot {IF}_{{AO}_{i}}}}} & (6)\end{matrix}$

Adapting Images on Small Displays

Given the image attention model, now let us consider how to adapt animage 210 to fit into a small screen which is often the major limitationof mobile devices. For purposes of discussion, such a mobile device orother client device is represented by computing device 130 of FIG. 1 orremote computing device 182, which is also located in FIG. 1. The smallscreen is represented in this example via display 176 or 190 of FIG. 1.It can be appreciated that when the computing device is a handheld,mobile, or other small footprint device that the display size may bemuch smaller (e.g., several centimeters in diameter) than a computermonitor screen. We address the problem of making the best use of atarget area T to represent images while maintaining their originalspatial ratios. Various image adaptation schemes can be applied toobtain different results. For each adapted result, there is acorresponding unique solution which can be presented by a region R inthe original image. In other words, an adapted result is generated fromthe outcome of scaling down its corresponding region R. As screen sizeis our main focus, we assume the color depth and compression qualitydoes not change in our adaptation scheme.

FIG. 6 shows that image attention objects (AOs) are segmented intoregions and respectively adapted to presentation characteristics of aclient display target area. For purposes of discussion, the image 600 isone of the images 210-1 through 210-K of FIG. 2, and the AOs 602represent respective ones of the AOs 212-1 through 212-N of FIG. 2. Inthis example, image regions include region 604 and region 606. Region604 encapsulates AOs 602-1 and 602-2 and has a height R₁ and width R₁.Region 606 encapsulates AOs 602-2 and 602-K and has a height R₂ andwidth R₂. For purposes of discussion, regions 604 and 606 are shown asrespective rectangles. However, in other implementations a region is nota rectangle but some other geometry.

In this example, region 604 has been adapted to region 608 by thecontent adaptation module 202 of FIG. 2, and region 606 has been adaptedto region 610. In both cases of this example, note that the adaptedregion is dimensionally smaller in size than its corresponding parentregion. However, as a function of the particular characteristics of theimage and the target area of the client device, it is possible for anadapted region to be larger than the regions from which it was adapted.In other words, each region of an image is adapted as a function of thespecific target area of the client device that is going to be used topresent the content and the desired image fidelity, which is expressedas follows:

According to Equation (6), an objective measure for the informationfidelity of an adapted image is formulated as follows:

$\begin{matrix}\begin{matrix}{{IF}_{R} = {\sum\limits_{{ROI}_{i} \Subset R}{{AV}_{i} \cdot {IF}_{{AO}_{i}}}}} \\{= {\sum\limits_{{ROI}_{i} \Subset R}{{AV}_{i} \cdot {u\left( {{r_{R}^{2} \cdot {{size}\left( {ROI}_{i} \right)}} - {MPS}_{i}} \right)}}}}\end{matrix} & (7)\end{matrix}$where u(x) is defined as

${u(x)} = \left\{ {\begin{matrix}{1,} & {{{if}\mspace{14mu} x} \geq 0} \\{0,} & {otherwise}\end{matrix}.} \right.$Function size (x) calculates the area of a ROI, and r_(R) denotes theratio of image scaling down, which is computed as

$\begin{matrix}{r_{R} = {\min\left( {\frac{{Width}_{T}}{{Width}_{R}},\frac{{Height}_{T}}{{Height}_{R}}} \right)}} & (8)\end{matrix}$Width_(T), Height_(T), Width_(R), and Height_(R) represent the widthsand heights of target area T and solution region R, respectively. Asshown in FIG. 6, when adapting an image to different target areas, theresulting solution regions may be different.

This quantitative value is used to evaluate all possible adaptationschemes to select the optimal one, that is, the scheme achieving thelargest IF value. Taking the advantage of our image attention model, wetransform the problem of making adaptation decision into the problem ofsearching a region within the original image that contains the optimalAO set (i.e. carries the most information fidelity), which is defined asfollows:

$\begin{matrix}{\max\limits_{R}\left\{ {\sum\limits_{{ROI}_{i} \Subset R}{{AV}_{i} \cdot {u\left( {{r_{R}^{2} \cdot {{size}\left( {ROI}_{i} \right)}} - {MPS}_{i}} \right)}}} \right\}} & (9)\end{matrix}$

The Image Adaptation Algorithm

For an image 210 (FIG. 2) with width m and height n, the complexity forfinding the optimal solution of (9) is O(m²n²) because of the arbitrarylocation and size of a region. Since m and n may be quite large, thecomputational cost could be expensive. However, since the informationfidelity of adapted region is solely decided by its attention objects212, we can greatly reduce the computation time by searching the optimalAO set before generating the final solution.

Determining a Valid Attention Object Set

We introduce I as a set of AOs, I⊂{AO₁, AO₂, . . . AO_(N)}. Thus, thefirst step of optimization is to find the AO set that carries thelargest information fidelity after adaptation. Let us consider R_(I),the tight bounding rectangle containing all the AOs in I. We can firstadapt R_(I) to the target area T, and then generate the final result byextending R_(I) to satisfy the requirements.

All of the AOs within a given region R may not be perceptible whenscaling down R to fit a target area T. Thus, to reduce the solutionspace, an attention object set is valid if:

$\begin{matrix}\begin{matrix}{{\frac{{MPS}_{i}}{{size}\left( {ROI}_{i} \right)} \leq r_{I}^{2}},} & {{\forall{{AO}_{i} \in I}},}\end{matrix} & (10)\end{matrix}$wherein r_(I) (r_(I) is equivalent to r_(R) _(I) in Equation (8) forsimplicity) is the ratio of scaling down when adapting the tightbounding rectangle R_(I) to T, which can be computed as follows:

$\begin{matrix}\begin{matrix}{r_{I} = {\min\left( {\frac{{Width}_{T}}{{Width}_{I}},\frac{{Height}_{T}}{{Height}_{I}}} \right)}} \\{= {{\min\left( {\frac{{Width}_{T}}{\max\limits_{{AO}_{i},{{AO}_{j} \in I}}{{{Right}_{i} - {Left}_{j}}}},\frac{{Height}_{T}}{\max\limits_{{AO}_{i},{{AO}_{j} \in I}}{{{Bottom}_{i} - {Top}_{j}}}}} \right)}.}}\end{matrix} & (11)\end{matrix}$Herein, Width_(I) and Height_(I) denote the width and height of R_(I),while Left_(i), Right_(i), Top_(i), and Bottom_(i) are the four boundingattributes of the i^(−th) attention object.

r_(I) in equation 10 is used to check scaling ratio, which should begreater than √{square root over (MPS_(i)/size(ROI_(i)))} for any AO_(i)belonging to a valid I. This ensures that all AO included in I isperceptible after scaled down by a ratio r_(I). For any two AO sets I₁and I₂, there has r_(I) ₁ ≧r_(I) ₂ , if I₁⊂I₂. Thus, it isstraightforward to infer the following property of validity fromequation 10. If I₁⊂I₂ and I₁ is invalid, then I₂ is invalid (property1).

With the definition of valid attention object set, the problem ofEquation (9) is further simplified as follows:

$\quad\begin{matrix}\begin{matrix}{{\max\limits_{I}\left( {IF}_{I} \right)} = {\max\limits_{I}\left( {\sum\limits_{{AO}_{i} \in I}{{AV}_{i} \cdot {u\left( {{r_{I}^{2} \cdot {{size}\left( {ROI}_{i} \right)}} - {MPS}_{i}} \right)}}} \right)}} \\{= {\max\limits_{I}{\left( {\sum\limits_{{AO}_{i} \in I}{AV}_{i}} \right)\mspace{25mu}{\forall{{{valid}\mspace{14mu} I} \Subset \left\{ {{AO}_{1},{AO}_{2},{\ldots\mspace{11mu}{AO}_{N}}} \right\}}}}}}\end{matrix} & (12)\end{matrix}$As can be seen, this has become an integer programming problem and theoptimal solution is found via use of a branch and bound algorithm.

Branch and Bound Process

FIG. 7 shows a binary tree 700 to illustrate a branch and bound processto identify an optimal image adaptation solution. Each level 0-2 of thebinary tree includes different sets of AOs 212-1 through 212-N. Eachnode of the binary tree denotes zero or more specific sets of AOs. Eachbifurcation of the binary tree represents a decision/opportunity to keepor drop AO(s) of the next level. The height of the binary treecorresponds to K, the number of AOs inside the particular image (i.e.,one of the images 210-1 through 210-K of FIG. 2). Each leaf node in thistree corresponds a different possible I.

For each node in the binary AO tree 700, there is a boundary on thepossible IF value it can achieve among all of its sub-trees. The lowerboundary is just the IF value currently achieved when none of theunchecked AOs can be added (i.e., the sum of IF values of AOs includedin current configuration). The upper boundary is the addition of all IFvalues of those unchecked AOs after current level (i.e., the sum of IFvalues of all AOs in the image except those dropped before currentlevel).

Whenever the upper bound of a node is smaller than the best IF valuecurrently achieved, the whole sub-tree of that node is truncated. At thesame time, for each node we check the ratio r_(I) of its correspondingAO set I to verify its validity. If it is invalid, according to property1, the whole sub-tree of that node is also truncated. By checking boththe bound on possible IF value and the validity of each AO set, thecomputation cost is greatly reduced.

A number of techniques can be used to reduce binary tree 700 traversaltime. For instance, arranging the AOs are arranged in decreasing orderof their AVs at the beginning of search will decrease traversal times,since in most cases only a few AOs contribute the majority of IF value.Additionally, when moving to a new level k, it is determined whetherAO_(k) is already included in current configuration. If so, travel thebranch of keeping AO_(k) and prune the one of dropping AO_(k) and allsub-branches.

Transform to Adapted Solution

After finding the optimal AO set I_(opt), the content adaptation module202 of FIG. 2 generates different possible solutions according todifferent requirements by extending R_(I) _(opt) while keeping I_(opt)valid. For instance, if an image 210-1 through 210-K has some backgroundinformation which is not included in the attention model 208, theadapted result may present a region as large as possible by extendingR_(I) _(opt) . The scaling ratio of final solution region is

$r_{I_{opt}}^{\max} = {\max\limits_{{AO}_{i} \in I_{opt}}\left( {{MPS}_{i}/{{size}\left( {ROI}_{i} \right)}} \right)}$to keep I_(opt) valid as well as to obtain the largest area. Therefore,R_(I) _(opt) is extended to a region determined by

r_(I_(opt))^(max)and T, within the original image.

In other cases, the adapted images 220 may be more satisfactory withhigher resolution than larger area. To this end, R_(I) _(opt) isextended, while keeping the scaling ratio at r_(I) _(opt) instead of

r_(I_(opt))^(max).However, it is worth noticing that in this situation, the scaled versionof whole image will perhaps never appear in adapted results.

Sometimes a better view of an image 210, or portions thereof, can beachieved when a display screen is rotated by ninety (90) degree. In sucha case, I_(opt)′ carries more information than I_(opt). In this case, wecompare the result with the one for the rotated target area, and thenselect the better one as the final solution.

The complexity of this algorithm is exponential with the number ofattention objects 212 within an image 210. However, the describedtechniques are efficiently performed because the number of attentionobjects in an image is often less than a few dozen and the correspondingattention values 214 are always distributed quite unevenly amongattention objects.

An Exemplary Procedure

FIG. 8 shows an exemplary procedure 800 for enhanced image adaptation inview of client resource constraints. For purposes of discussion, theoperations of this procedure are described in reference to the programmodule and data components of FIGS. 1 and 2. At block 802, the contentadaptation module 202 (FIG. 2) models an image 210 modeled with respectto multiple visual attentions (e.g., saliency, face, text, etc.) togenerate respective attention objects (AOs) 212 for each of the visualattentions. At block 804, for each of one or more possible imageadaptation schemes (e.g., resolution reduction, compression, etc.), thecontent adaptation module determines an objective measure of informationfidelity (IF) for a region R of the image. The objective measures aredetermined as a function of a resource constraint of the display device(e.g., displays 172 or 190 of FIG. 1) and as a function of a weightedsum of IF of each AO in the region R. (Such objective measures andweighted sums are represented as respective portions of “other data” 222of FIG. 2). At block 806, the content adaptation module selects asubstantially optimal image adaptation scheme as a function of thecalculated objective measures. At block 808, the content adaptationmodule adapts the image via the selected substantially optimaladaptation scheme to generate an adapted image 220 (FIG. 2) as afunction of at least the target area of the client display.

CONCLUSION

The described systems and methods use attention modeling and otherobjective criteria for enhanced image adaptation in view of clientresource constraints. Although the systems and methods have beendescribed in language specific to structural features and methodologicaloperations, the subject matter as defined in the appended claims are notnecessarily limited to the specific features or operations described.Rather, the specific features and operations are disclosed as exemplaryforms of implementing the claimed subject matter.

1. A method for enhanced image adaptation for a display device having atarget area T, the method comprising: modeling an image with respect toa plurality of visual attentions to generate a respective set ofattention objects for each attention of the visual attentions; assigninga respective attention value to each attention object, the attentionvalue indicating a relative importance of image information representedby the attention object as compared to other ones of the attentionobjects; indicating a respective weight for each of the visualattentions; for each visual attention, adjusting assigned attentionvalues of corresponding attention objects based on corresponding visualattention weight, the adjusting comprising: normalizing the assignedattention values to a unit interval; and computing adjusted attentionvalues according to: AV_(i) = w_(k) ⋅ AV_(i)^(k), wherein AV_(i) is anadjusted attention value for attention object i derived from a visualattention model k, w_(k) is a weight of the visual attention model k,and AV_(i) ^(k) is a normalized attention value of attention objectAO_(i) detected for the visual attention model k; for each of one ormore image adaptation schemes, determining a respective objectivemeasure of information fidelity (IF) of a region R of the image as afunction of a resource constraint of the display device and as afunction of a weighted sum of IF of a value of each attention object(AO_(i)) in the region R; selecting a substantially optimal adaptationscheme from among the one or more image adaptation schemes, each imageadaptation scheme ranked as a function of, at least, its determinedobjective measure of information fidelity IF with respect to the regionR; and adapting the image via the substantially optimal adaptationscheme to generate an adapted image as a function of at least the targetarea T.
 2. A method as recited in claim 1, wherein the multiple visualattentions are based on saliency, face, and text attention models.
 3. Amethod as recited in claim 1, wherein each objective measure is uniqueas compared to each other objective measure(s).
 4. A method as recitedin claim 1, wherein resource constraint comprises one or more of spatialregion size (T), color depth, and/or ratio of compression.
 5. A methodas recited in claim 1, wherein the image is contained in compositecontent comprising other images, and wherein modeling the image furthercomprises determining a context of the image as compared to respectivecontexts of the other images to assist in determining attention objectattributes of the image.
 6. A method as recited in claim 1, wherein theweighted sum of information fidelity (IF) of each (i) attention object(AO) in the region R is based on:${{IF}_{R} = {\sum\limits_{{ROI}_{i} \Subset R}{{AV}_{i} \cdot {IF}_{{AO}_{i}}}}};$and wherein ROI_(i) is a spatial size of AO_(i).
 7. A method as recitedin claim 1, wherein selecting the substantially optimal adaptationscheme further comprises searching a region of the image for anattention object set that represents higher IF after adaptation ascompared to collective IF values of other attention object sets afteradaptation.
 8. A method as recited in claim 1, wherein selecting thesubstantially optimal adaptation scheme further comprises calculatingthe substantially optimal adaptation scheme via integer programming anduse of a branch and bound algorithm.
 9. A method as recited in claim 1,wherein selecting the substantially optimal adaptation scheme is basedon the following: $\begin{matrix}{{\max\limits_{I}\left( {IF}_{I} \right)} = {\max\limits_{I}\left( {\sum\limits_{{AO}_{i} \in I}{{AV}_{i} \cdot {u\left( {{r_{I}^{2} \cdot {{size}\left( {ROI}_{i} \right)}} - {MPS}_{i}} \right)}}} \right)}} \\{{= {\max\limits_{I}{\left( {\sum\limits_{{AO}_{i} \in I}{AV}_{i}} \right)\mspace{25mu}{\forall{{{valid}\mspace{14mu} I} \Subset \left\{ {{AO}_{1},{AO}_{2},{\ldots\mspace{14mu}{AO}_{N}}} \right\}}}}}};{and}}\end{matrix}$ wherein I is a set of AOs, ROI is a spatial size ofAO_(i), and MPS_(i) is a minimal perceptible size beyond which AO_(i) isnot to be reduced.
 10. A method as recited in claim 1, wherein adaptingthe image via the substantially optimal adaptation scheme to generate anadapted image further comprises iteratively scaling the region R to aregion r comprising a determined set of optimal attention objects(I_(opt)) by extending and/or contracting the region while keepingI_(opt) valid.
 11. A method as recited in claim 1, wherein adapting theimage further comprises scaling the region R to a region r comprising adetermined set of optimal attention objects (I_(opt)) according to thefollowing scaling equation:${r_{I_{opt}}^{\max} = {\max\limits_{{AO}_{i} \in I_{opt}}\left( {{MPS}_{i}/{{size}\left( {ROI}_{i} \right)}} \right)}};$wherein MPS_(i) is the Minimal Perceptible Size of AO_(i), whereinROI_(i) is a spatial area of AO_(i) in the image; and wherein thesealing equation is solved such that I_(opt) remains valid and such thatthe region r is scaled to a largest region within target area T.
 12. Amethod as recited in claim 1, wherein adapting the image furthercomprises seating the region R comprising a determined set of optimalattention objects (I_(opt)) such that I_(opt) remains valid and suchthat the region R is scaled to a highest resolution to fit within targetarea T.
 13. A method as recited in claim 1, wherein the weighted sum ofinformation fidelity (IF) of each (i) attention object (AO) in theregion R is based on: $\quad\begin{matrix}{{IF}_{R} = {\sum\limits_{{ROI}_{i} \Subset R}{{AV}_{i} \cdot {IF}_{{AO}_{i}}}}} \\{{= {\sum\limits_{{ROI}_{i} \Subset R}{{AV}_{i} \cdot {u\left( {{r_{R}^{2} \cdot {{size}\left( {ROI}_{i} \right)}} - {MPS}_{i}} \right)}}}};}\end{matrix}$ wherein u(x) is defined as${u(x)} = \left\{ {\begin{matrix}{1,} & {{{if}\mspace{14mu} x} \geq 0} \\{0,} & {otherwise}\end{matrix};} \right.$  and wherein ROI_(i) is a spatial size ofAO_(i), wherein size(x) calculates the spatial size, and r_(R) denotes aratio of image scaling down.
 14. A method as recited in claim 13,wherein the ratio of image scaling down is computed as follows:${r_{R} = {\min\left( {\frac{{Width}_{T}}{{Width}_{R}},\frac{{Height}_{T}}{{Height}_{R}}} \right)}};$and wherein Width_(T), Height_(T), Width_(R), and Height_(R) representwidths and heights of target area T and solution region R, respectively.15. A computer-readable medium storing computer-executable instructionsfor enhanced image adaptation for a display device having a target areaT, the computer-executable instructions comprising instructions for:modeling an image with respect to a plurality of visual attentions togenerate a respective set of attention objects for each attention of thevisual attentions; for each of one or more image adaptation schemes,determining a respective objective measure of information fidelity (IF)of a region R of the image as a function of, at least: a resourceconstraint of the display device; and a weighted sum of an informationfidelity of each attention object (AO_(i)) in the region R based on:${{IF}_{R} = {\sum\limits_{{ROI}_{i} \Subset R}{{AV}_{i} \cdot {IF}_{{AO}_{i}}}}},$wherein IF_(R) corresponds to the weighted sum, ROI_(i) is a spatialsize of the attention object AO_(i), AV_(i) is a value of the attentionobject AO_(i), and IF_(AOi) is the information fidelity of the attentionobject AO_(i); selecting a substantially optimal adaptation scheme fromamong the one or more image adaptation schemes, each image adaptationscheme ranked as a function of, at least, its determined objectivemeasure of information fidelity IF with respect to the region R; andadapting the image via the substantially optimal adaptation scheme togenerate an adapted image as a function of at least the target area T.16. A computer-readable medium as recited in claim 15, wherein themultiple visual attentions are based on saliency, face, and textattention models.
 17. A computer-readable medium as recited in claim 15,wherein each objective measure is unique as compared to each otherobjective measure(s).
 18. A computer-readable medium as recited in claim15, wherein resource constraint comprises one or more of spatial regionsize (T), color depth, and/or ratio of compression.
 19. Acomputer-readable medium as recited in claim 15, wherein the image iscontained in composite content comprising other images, and wherein theinstructions for modeling the image further comprises determining acontext of the image as compared to respective contexts of the otherimages to assist in determining attention object attributes of theimage.
 20. A computer-readable medium as recited in claim 15, andwherein the computer-executable instructions further compriseinstructions for: assigning a respective attention value to eachattention object, the attention value indicating a relative importanceof image information represented by the attention object as compared toother ones of the attention objects; indicating a respective weight foreach of the visual attentions; and for each visual attention, adjustingassigned attention values of corresponding attention objects based oncorresponding visual attention weight.
 21. A computer-readable medium asrecited in claim 20, wherein the instructions for adjusting assignedattention values further comprise instructions for: normalizing theassigned attention values to (0, 1); and computing adjusted attentionvalues as follows:${{AV}_{i} = {w_{k} \cdot \overset{\_}{{AV}_{i}^{k}}}},$ wherein AV_(i)is an adjusted attention value for attention object i derived fromvisual attention model k, w_(k) is the weight of the visual attentionmodel k, and AV_(i) ^(k) is a normalized mention value of attentionobject AO_(i) detected in the visual attention model k.
 22. Acomputer-readable medium as recited in claim 15, wherein theinstructions for selecting the substantially optimal adaptation schemefurther comprise instructions for searching a region of the image for anattention object set that represents higher IF after adaptation ascompared to collective IF values of other attention object sets afteradaptation.
 23. A computer-readable medium as recited in claim 15,wherein the instructions for selecting the substantially optimaladaptation scheme further comprise instructions for calculating thesubstantially optimal adaptation scheme via integer programming and useof a branch and bound algorithm.
 24. A computer-readable medium asrecited in claim 15, wherein the instructions for selecting thesubstantially optimal adaptation scheme is based on the following:$\begin{matrix}{{\max\limits_{I}\left( {IF}_{I} \right)} = {\max\limits_{I}\left( {\sum\limits_{{AO}_{i} \in I}{{AV}_{i} \cdot {u\left( {{r_{I}^{2} \cdot {{size}\left( {ROI}_{i} \right)}} - {MPS}_{i}} \right)}}} \right)}} \\{{= {\max\limits_{I}{\left( {\sum\limits_{{AO}_{i} \in I}{AV}_{i}} \right)\mspace{25mu}{\forall{{{valid}\mspace{14mu} I} \Subset \left\{ {{AO}_{1},{AO}_{2},{\ldots\mspace{14mu}{AO}_{N}}} \right\}}}}}};{and}}\end{matrix}$ wherein I is a set of AOs, ROI is a spatial size ofAO_(i), and MPS_(i) is a minimal perceptible size beyond which AO_(i) isnot to be reduced.
 25. A computer-readable medium as recited in claim15, wherein the instructions for adapting the image further compriseinstructions for iteratively scaling the region R to a region rcomprising a determined set of optimal attention objects (I_(opt)) byextending and/or contracting the region while keeping I_(opt) valid. 26.A computer-readable medium as recited in claim 15, wherein theinstructions for adapting the image further comprise instructions forscaling the region R to a region r comprising a determined set ofoptimal attention objects (I_(opt)) according to the following scalingequation:${r_{I_{opt}}^{\max} = {\max\limits_{{AO}_{i} \in I_{opt}}\left( {{MPS}_{i}/{{size}\left( {ROI}_{i} \right)}} \right)}};$wherein MPS_(i) is the Minimal Perceptible Size of AO_(i), whereinROI_(i) is a spatial area of AO_(i) in the image; and wherein thescaling equation is solved such that I_(opt) remains valid and such thatthe region r is scaled to a largest region within target area T.
 27. Acomputer-readable medium as recited in claim 15, wherein theinstructions for adapting the image further comprise instructions forscaling the region R comprising a determined set of optimal attentionobjects (I_(opt)) such that I_(opt) remains valid and such that theregion R is scaled to a highest resolution to fit within target area T.28. A computer-readable medium as recited in claim 15, wherein theweighted sum of information fidelity (IF) of each (i) attention object(AO) in the region R is based on: $\begin{matrix}{{IF}_{R} = {\sum\limits_{{ROI}_{i} \Subset R}{{AV}_{i} \cdot {IF}_{{AO}_{i}}}}} \\{{= {\sum\limits_{{ROI}_{i} \Subset R}{{AV}_{i} \cdot {u\left( {{r_{R}^{2} \cdot {{size}\left( {ROI}_{i} \right)}} - {MPS}_{i}} \right)}}}};}\end{matrix}$ wherein u(x) is defined as${u(x)} = \left\{ {\begin{matrix}{1,} & {{{if}\mspace{14mu} x} \geq 0} \\{0,} & {otherwise}\end{matrix};} \right.$  and wherein ROI_(i) is a spatial size ofAO_(i), wherein size(x) calculates the spatial size, and r_(R) denotes aratio of image scaling down.
 29. A computer-readable medium as recitedin claim 28, wherein the ratio of image scaling down is computed asfollows:${r_{R} = {\min\left( {\frac{{Width}_{T}}{{Width}_{R}},\frac{{Height}_{T}}{{Height}_{R}}} \right)}};$and wherein Width_(T), Height_(T), Width_(R), and Height_(R) representwidths and heights of target area T and solution region R, respectively.30. A computing device for enhanced image adaptation for a displaydevice having a target area T, the computing device comprising: aprocessor; and a memory coupled to the processor, the memory comprisingcomputer-program executable instructions executable by the processorfor, at least: modeling an image with respect to a plurality of visualattentions to generate a respective set of attention objects for eachattention of the visual attentions; for each of one or more imageadaptation schemes, determining a respective objective measure ofinformation fidelity (IF) of a region R of the image as a function of,at least: a resource constraint of the display device; and a weightedsum of an information fidelity of each attention object (AO_(i)) in theregion R; selecting a substantially optimal adaptation scheme from amongthe one or more image adaptation schemes, each image adaptation schemeranked as a function of, at least, its determined objective measure ofinformation fidelity IF with respect to the region R; and adapting theimage via the substantially optimal adaptation scheme to generate anadapted image as a function of at least the target area T, the adaptingcomprising: scaling the region R to a region r comprising a determinedset of optimal attention objects (I_(opt)) according to the followingscaling equation:${r_{I_{opt}}^{\max} = {\max\limits_{{AO}_{i} \in I_{opt}}\left( {{MPS}_{i}/{{size}\left( {ROI}_{i} \right)}} \right)}};$wherein MPS_(i) is a minimum perceptible size of AO_(i), and ROI_(i) isa spatial area of AO_(i) in the image; and wherein the scaling equationis solved such that I_(opt) remains valid and such that the region r isscaled to a largest region within the target area T.
 31. A computingdevice as recited in claim 30, wherein the multiple visual attentionsare based on saliency, face, and text attention models.
 32. A computingdevice as recited in claim 30, wherein each objective measure is uniqueas compared to each other objective measure(s).
 33. A computing deviceas recited in claim 30, wherein resource constraint comprises one ormore of spatial region size (T), color depth, and/or ratio ofcompression.
 34. A computing device as recited in claim 30, wherein theimage is contained in composite content comprising other images, andwherein the instructions for modeling the image further comprisesdetermining a context of the image as compared to respective contexts ofthe other images to assist in determining attention object attributes ofthe image.
 35. A computing device as recited in claim 30, and whereinthe computer-executable instructions further comprise instructions for:assigning a respective attention value to each attention object, theattention value indicating a relative importance of image informationrepresented by the attention object as compared to other ones of theattention objects; indicating a respective weight for each of the visualattentions; and for each visual attention, adjusting assigned attentionvalues of corresponding attention objects based on corresponding visualattention weight.
 36. A computing device as recited in claim 35, whereinthe instructions for adjusting assigned attention values furthercomprise instructions for: normalizing the assigned attention values to(0, 1); and computing adjusted attention values as follows:${{AV}_{i} = {w_{k} \cdot \overset{\_}{{AV}_{i}^{k}}}},$ wherein AV_(i)is an adjusted attention value for attention object i derived fromvisual attention model k, w_(k) is the weight of the visual attentionmodel k, and AV_(i) ^(k) is a normalized attention value of attentionobject AO_(i) detected in the visual attention model k.
 37. A computingdevice as recited in claim 30, wherein the weighted sum of informationfidelity (IF) of each (i) attention object (AO) in the region R is basedon:${{IF}_{R} = {\sum\limits_{{ROI}_{i} \Subset R}{{AV}_{i} \cdot {IF}_{{AO}_{i}}}}};$and wherein ROI_(i) is a spatial size of AO_(i).
 38. A computing deviceas recited in claim 30, wherein the instructions for selecting thesubstantially optimal adaptation scheme further comprise instructionsfor searching a region of the image for an attention object set thatrepresents higher IF after adaptation as compared to collective IFvalues of other attention object sets after adaptation.
 39. A computingdevice as recited in claim 30, wherein the instructions for selectingthe substantially optimal adaptation scheme further compriseinstructions for calculating the substantially optimal adaptation schemevia integer programming and use of a branch and bound algorithm.
 40. Acomputing device as recited in claim 30, wherein the instructions forselecting the substantially optimal adaptation scheme is based on thefollowing: $\begin{matrix}{{\max\limits_{I}\left( {IF}_{I} \right)} = {\max\limits_{I}\left( {\sum\limits_{{AO}_{i} \in I}^{\;}\;{{AV}_{i} \cdot {u\left( {{r_{I}^{2} \cdot {{size}\left( {ROI}_{i} \right)}} - {MPS}_{i}} \right)}}} \right)}} \\{{= {\max\limits_{I}{\left( {\sum\limits_{{AO}_{i} \in I}^{\;}\;{AV}_{i}} \right)\mspace{31mu}{\forall\mspace{14mu}{{{valid}\mspace{14mu} I} \Subset \left\{ {{AO}_{1},{AO}_{2},{\ldots\mspace{11mu}{AO}_{N}}} \right\}}}}}};{and}}\end{matrix}$ wherein I is a set of AOs, ROI is a spatial size ofAO_(i), and MPS_(i) is a minimal perceptible size beyond which AO_(i) isnot to be reduced.
 41. A computing device as recited in claim 30,wherein the instructions for adapting the image further compriseinstructions for iteratively scaling the region R to a region rcomprising a determined set of optimal attention objects (I_(opt)) byextending and/or contracting the region while keeping I_(opt) valid. 42.A computing device as recited in claim 30, wherein the instructions foradapting the image further comprise instructions for scaling the regionR comprising a determined set of optimal attention objects (I_(opt))such that I_(opt) remains valid and such that the region R is scaled toa highest resolution to fit within target area T.
 43. A computing deviceas recited in claim 30, wherein the weighted sum of information fidelity(IF) of each (i) attention object (AO) in the region R is based on:$\begin{matrix}{{IF}_{R} = {\sum\limits_{{ROI}_{i} \Subset R}^{\;}\;{{AV}_{i} \cdot {IF}_{{AO}_{i}}}}} \\{{= {\sum\limits_{{ROI}_{i} \Subset R}^{\;}\;{{AV}_{i} \cdot {u\left( {{r_{R}^{2} \cdot {{size}\left( {ROI}_{i} \right)}} - {MPS}_{i}} \right)}}}};}\end{matrix}$ wherein u(x) is defined as${u(x)} = \left\{ {\begin{matrix}{1,} & {{{if}\mspace{14mu} x} \geq 0} \\{0,} & {otherwise}\end{matrix};{and}} \right.$ wherein ROI_(i) is a spatial size ofAO_(i), wherein size(x) calculates the spatial size, and r_(R) denotes aratio of image scaling down.
 44. A computing device as recited in claim43, wherein the ratio of image scaling down is computed as follows:${r_{R} = {\min\begin{pmatrix}{\frac{{Width}_{T}}{{Width}_{R}},} & \frac{{Height}_{T}}{{Height}_{R}}\end{pmatrix}}};\mspace{14mu}{and}$ wherein Width_(T), Height_(T),Width_(R), and Height_(R) represent widths and heights of target area Tand solution region R, respectively.
 45. A computer-implemented systemfor enhanced image adaptation for a display device having a target areaT, the system comprising one or more program modules configured to, atleast: model an image with respect to a plurality of visual attentionsto generate a respective set of attention objects for each attention ofthe visual attentions; determine, for each of one or more imageadaptation schemes, a respective objective measure of informationfidelity (IF) of a region R of the image as a function of, at least: aresource constraint of the display device; and a weighted sum of aninformation fidelity of each attention object (AO_(i)) in the region R;select a substantially optimal adaptation scheme from among the one ormore image adaptation schemes, each image adaptation scheme ranked as afunction of, at least, its determined objective measure of informationfidelity IF with respect to the region R, and the selecting based on thefollowing: $\quad\begin{matrix}{{\max\limits_{I}\left( {IF}_{I} \right)} = {\max\limits_{I}\left( {\sum\limits_{{AO}_{1} \in I}{{AV}_{i} \cdot {u\left( {{r_{I}^{2} \cdot {{size}\left( {ROI}_{i} \right)}} - {MPS}_{i}} \right)}}} \right)}} \\{{= {\max\limits_{I}{\left( {\sum\limits_{{AO}_{1} \in I}{AV}_{i}} \right){\forall{{{valid}\mspace{14mu} I} \Subset \left\{ {{AO}_{1},{AO}_{2},{\ldots\mspace{20mu}{AO}_{N}}} \right\}}}}}},}\end{matrix}$ wherein I is a set of valid attention objects, IF_(I) isan information fidelity of I, AV_(i) is an attention value of AO_(i),r_(I) is a scaling ratio for an image corresponding to I, size(ROI_(i))is a spatial size of a region ROI_(i) corresponding to AO_(i), MPS_(i)is a minimal perceptible size of AO_(i), and the function u(x) has thevalue 1 when x is nonnegative and 0 otherwise; and adapt the image viathe substantially optimal adaptation scheme to generate an adapted imageon an output display as a function of at least the target area T. 46.The system of claim 45, wherein at least one attention of the visualattentions is based on at least one of: a saliency attention model, aface attention model, and a text attention model.
 47. The system ofclaim 45, wherein the resource constraint comprises at least one ofspatial region size, color depth, and ratio of compression.
 48. Thesystem of claim 45, wherein: the image is contained in a compositecontent comprising other images; and modeling the image furthercomprises determining a context of the image as compared to respectivecontexts of the other images to assist in determining attention objectattributes of the image.
 49. The system of claim 45, wherein the one ormore program modules are further configured to, at least: assign arespective attention value to each attention object, the attention valueindicating a relative importance of image information represented by theattention object as compared to other ones of the attention objects;indicate a respective weight for each of the visual attentions; and foreach visual attention, adjust assigned attention values of correspondingattention objects based on corresponding weights.